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19 February 1982 


MEMORANDUM FOR: Director of Central Intelligence 
Director, Intelligence Community Staff 
Deputy Director of Central Intelligence 


SUBJECT: Project SAFE 


‘1. I am forwarding the memorandum prepared by[ ner STAP STAT 
on his discussions with Government personnel and oncerning the SAFE STAT 
project. I concur with his conclusions, options, and recommendations, based 


on my own independent assessments, having discussed the matter with officials 
at CIA, and DIA. 


2, While STAP through the accompanying memorandum presents two options, 
jt is our firm belief that the SAFE contract should be terminated at once. In 
coming to this conclusion, we recognize the downside dangers of prejudicing 
further work on systems that are urgently needed by the analyst. Attempting 
to save a flawed system by spending two million dollars a month is not a wise 
investment. Much has been learned from the SAFE experience, but a system 
whose architecture is based on early 1970's technology is not what will be 
needed in the late 1980's. A new start, incorporating the lessons learned 
from SAFE, involving intimate interactions between analysts and the systems 
architect office based on a contractural arrangement which requires on-site 
participation by all parties, and which is monitored on a continuing basis by 
outside experts, is a much sounder investment. 


3. In reaching this conclusion, we recognized, as is stated in the 
memorandum, that considerations other than those of a technical or managerial 
nature may make termination politically jmpossible. Nonetheless we strongly 
urge you to consider termination. 
STAT 
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Subject: SAFE 


0.0 ABSTRACT 


This memorandum analyzes the current status of the SAFE project, 
assesses its future, evaluates certain options, and makes: 
recommendations about what to do. The immediate problem is that 
the current contractor's| __ proposal for what wilt be 
initially delivered not only is undefined in many particulars, 
but also is simply unacceptable to the Government; to correct 
that will surely take both money and time. We must make certain 
that the SAFE system provided will be as good as possible. 


The deficiencies in the situation are that: 1) the designers at 
ave never really been aware of the operational environment 
in which SAFE will work; 2) the design is unfinished in many 
details, including some important major decisions, like the 
distribution of functionality between the| | mainframes 


and the user terminals; 3) the user language reflects almost none 
of the lengthy detailed work that SAS and CSPO have undertaken -- 
but note that has recently agreed to accept the Government's 

set -of language commands; a[ jis responding only to the stated 
requirements, with little appreciation of the demanding jobs that 


analysts have; 5) it is not clear-whether the hardware will 


support both the projected usage and the planned number of users; 


6) it is not clear whether the other design decisions are 
appropriate. 


We recommend that all elements of the design be reassessed by 


’ independent experts. If these evaluations are favorable, the 


Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2 


STAT 


STAT 


2/19/82 OFFICIAL USE ONLY Page ii 
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2 


appropriate design decisions should be made to correct known. 
deficiencies and to resolve the many major open issues, and the 
contractor should be tasked to produce a full Block 1 capability. 
Enough time should be allowed for this -- we estimate it will 
take 18 months more than now scheduled, at least. It will also 
take more money. The alternative, we recommend, is to terminate 
the contract, though not the effort to provide analysts with 
SAFE-like capabilities. Furthermore, we recommend in any event 
that the Pilot Mail Operation be extended and enhanced. 
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1.0 INTRODUCTION 


This memo is part of a continuing review of the SAFE project. . STAT 
SAFE is. entering a crucial period, expenditures [ _ ere rising STAT 
to over $2M a month, and the proposed initial deliverable 

seems to be unacceptable to the Agency and DIA. It seems clear 

that soon certain decisions must be made about modifications and 

delays in the contract that will have a large impact jin both 

Agencies. Here I discuss the implications of the continuing 

problems, especially in the light of the late January 1982 

Preliminary Design Review (PDR), which I attended STAT 
with the cooperation of the Consolidated SAFE Project Office 

(CSPO) [ I am grateful to CSPO for making that possible 

and was impressed by the diligence and dedication of the 

government personnel at that meeting. | 


The history of the sAFE[  kontract is long and complicated, and 
I shall not try to summarize it here; there are, however, some 
pieces of it that must be mentioned: 


fe) By now, the presently projected early blocks will have far 
fewer capabilities than had been originally specified. As 
_ costs rose, because of changes, delays, etc., many 
capabilities were delayed, and some were dropped. This was 
because the SAFE contract was essentially fixed price, 
although what was Hever ieted was cost plus fee. 
. PeW 86 
0 In the summer (1981,-the [ hardware was selected. STAT 


an oo 


) The Initial Operating Capability (10C) had always been set 
for 1 Jan 83; it is now set at 14 Mar 83, apparently as a 
result of PERT projections. The initial deliverable is now 
termed ‘increment 2.5. 


0 In the summer and fall of 1981, it. was becoming increasingly 
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apparent to CSPO that the contract was in trouble. The 
aggregate Award Fee Scoring had fallen from about 75 to less 
than 40 in the 5th 6-month period, and the "overall 
performance for the 5th period was evaluated as marginal."* 


In the fall of 1981, partly as a result of that assessment, 

a significant reorganization of the staff was undertaken STAT 
("The ritual beheading," according to head of STAT 
the IC staff). 


In the middle of January 82, CSPO and the Systems Analysis 
Staff (SAS) of the Office of Central Referencé (OCR): learned 

was proposing to supply, with the first release of 
the deliverable system, a users' command language that 
seemed to reflect very few of the deliberations that the 
Government had been sharing with the corresponding group on 
Language Development| _—_—i| The functionality that was seen 
jn the proposed release seemed dubious, and did not match . 
what SAS had been discussing, __— called the proposed 2.5 STAT 
delivery "abysmal", and CSPO in general regarded it as 
unacceptable. In fact, none of the eventual user languages 
have been fully specified, and for some, like the analysts’ 
editing’ language for compose,[ | submitted no proposals STAT 


at all. 


The PILOT Mail Operation (PMO), which was one of the 
successors to the Interim SAFE, has been regarded as 
successful enough to be considered as worth expanding (Bruce 
Johnson, ODP), if there are any substantial delays in SAFE 
jmplementation. : 


These and other topics will be explored below. Decisions. must be 


made very soon about what can be reasonably expected for the 


_*Memo: Point Paper for SAFE Steering Committees[ 
(Director of CSPO), 11/27/81. 


STAT 
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performance of SAFE, and to what schedule. If there are to be 
significant changes (and I am sure there will have to be), then: 
the political and financial implications may be very great, and 
it would be well to be sure about all the projections. 


There are many lessons to be learned by the Community from SAFE, 
and I shall try to discuss some of them. Many of them are far 
beyond the responsibility of the current participants in SAFE, 
and, indeed, some deal with the entire nature of the . 
relationships between Government and Industry in large and 
complex purchases like this one. Sometimes it must be appropriate 
to delegate the responsibility for overall and detailed design of 
a large and complex system, but it was not for SAFE. We will 
discuss this point further in section 4. | | 


But the most important recommendations are those that concern the 
immediate options that the government has. The need for a SAFE- 
like capability is enormous, and I believe that it would be a 
devastating loss to the Community should the effort ta achieve it 
be abandoned. The question is how to redirect the current thrust 
of the effort so that the best capabilities can be provided to 
the Community as quickly as possible. That will mean a speedy 
inspection of the proposed SAFE system in its current condition 
to make sure that it can become the basis for those capabilities; 
or else a study of how best to use the lessons learned. 


One of the difficulties in making recommendations is that there 
are very many factors and design decisions that have not yet been 
made, or at least been agreed upon, by both the contractor and 
the Government. For example, one of the fundamental capabilities 
to be provided to the analyst by SAFE is COMPOSE,* which deals 
with the building of a document to be produced as a finished 


*There are four major capabilities that SAFE should provide: 1) 
receive and distribute cable traffic; 2) create and maintain 

- analysts’ own files; 3) search SAFE files; 4) write or compose 
reports etc. é 
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piece of intelligence -- including editing, formatting, etc. 
Almost nothing about COMPOSE has been agreed upon. Indeed, the 
contractor has not even made serious proposals about what it 
might include, and neither has the government proposed a truly 
firm set of requirements to be discussed with the contractor. 


At. best, SAFE as currently forecast by both the contractor and 
CSPO will fall far short of what was originally envisioned by 
those responsible for its initiation, at least in the current 
contract. I believe that it is important to keep the vision 
alive, both as a goal to be striven for, and also as a key to 
keep us from accepting as a frozen design something that: will not 


‘Jet us reach it. 


2.0 CSPO CONCERNS 


There are a number of concerns that worry CSPO now; some have a 
long history, and some have arisen only recently. 


2.1 Management 


CSPO has always tried to ensure that it maintained control over 
communications[ _——__—=ijitt established that any contact with 
had to be made through CSPO. One problem that followed from that 
rule was that it was very difficult to observe the 
analysts working in their own environment on their own problems, 
so that [of what it was that analysts were doing was 
derived from the words in the specifications for SAFE -- and the 
specifications are not finished yet. It is apparently also true 
that jd not insist either in their proposal or so far during 
the contract.that such observations were essential not only to 
good human factors but also to a good balance of the 
functionality and the resources in SAFE with respect to present 
and future usage. 
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One healthy change in the CSPO viewpoint about management was to 
require Po lenstaniiee modifiability and flexibility in the 
hardware configurations and programming; so that when changes and 
modifications and extensions become clearly desirable, there will 
be but little trauma and interruption in making them. I am not 


sure to what extent this requirement has actually been put into 
effect, and that point should be carefully checked. 


The management of large and complex systems, especially when they 
involve software, is always remarkably difficult. Indeed, the 
acquisition of large software systems is probably the least 
successful of government contractual exercises.* CSPO early 
established an evaluation technique for this large contract (ca. 
$75M) called Award Fee Scoring: j 


fe) Technical Performance 50% 
i) Management 25% 
o Schedule 20% 
) Security | 5% 


In the Point Paper, [ 


"Overall performance for 5th (six-month) period was 
evaluated as marginal. 


A Problems outlined in previous evaluations persisted 
with predictable results 


i Management was aimed primarily toward gaining technical 
control -- ongoing activities were marginally managed. 


*See, for example, the Final Report of the Software Acquisition 


and Development Working Group, Chairman prepared for25X1 
the Assistant Secretary of Defense for Cwl, July 980. 
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‘ Management and organizational changes late in the i 
period place competent management in several key 
positions; results should be evident in upcoming 
periods." (ibid, p. 3) 


It seems clear from that statement that CSPO believed that the 
management and organizational problems[ bn this contract STAT 
were of long standing. Indeed, in the final minutes of the POR, 

(CSPO0) spoke of “knowing what the problems are,” 

hat "the level of uncertainty is down;| _ ad told STAT 

that the "SAFE Project is back on track."* A tough 

manager, had been named in late September as Deputy | 

Project Manager for Engineering, and seemed to be more effective. 


Much of the reassurance that CSPO was expressing in December was 
due to the PERTing [had initiated less than a year 
previously (chiefly as a result of pressure from CSPO), so that 

[ ___eoura report that the "Contractor (was) finally achieving 
detailed planning necessary for control and management of 
software development." (Point Paper, ibid, p. 5) 


That reassurance was badly shaken when the SAFE User Language 
(SUL) [ proposed to incorporate in the first release 
(increment 2.5, now scheduled on 3/14/83) was evaluated by SAS in 
1/82 as totally unacceptable, as lI mentioned above. Furthermore 
the functionality underlying that language, and which was the . 
rationale for it, seemed dubious indeed and not responsive to the 
user needs as they had been stated. The next section will discuss 


those topics. 


2.2 The SUL 


STAT 


‘= "The SAFE Project,": Briefing for| 
1/15/82, chart 3. 
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It is not overstating it to say that as far as the analysts are 
concerned the most important parts of SAFE are the ways they 
communicate with it; how they get it to do what they want done, 
and how they receive the information and products that they want. 
The SUL consists of a set of commands, in effect, which are input 
in several ways by users. The interactions of the Agencies with 


STAT jin the matter of user languages has been almost entirely 
through the SAS/CSPO User Interface Working Group (UIWG), which 
had believed that steady progress was being made. As late as 
11/81, the SAS reported that 
"(in order) to assure an orderly introduction of SAFE into 
NFAC on 31 December 1982 ... : 
"The User Language Specification (ULS) is to be in final 
form by the end of CY81."# 

In the middie of January 1982, a CSPO/SAS technical team reviewed 

SAFE documentation for the SUL. There had been essentially three 

languages proposed: 

) The government version, basically worked aut by the SAS/CSPO 
UIWG; a functional set of commands. Many of them had been 
delayed for Tater blocks as costs rose and delays occurred. 

STAT fo) The language team had been working with the 


Government, but was much more concerned with the problems of 

; implementation. For example, a key problem in designing SAFE 

STAT . is that both the [ si ecru's and the DeltaData terminal 

(7260T's) are capable of many of the functions required; how 
to separate the responsibilities so as to maximize the 
performance was of much more concern] | than to the STAT 
Government. This led to different emphases and different 
solutions to many problems. 


- #PUC No. 13, "The Plan for Implementing SAFE in NFAC," 11/4/81, 
p. 4. 
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0 The System Development a deriving language STAT 
specifications from the functional analysis of the CPU 
capabilities. 


The CSPO/SAS discovered by studying the commands that the third 

team was in effect the winner, had stated that the design was STAT 
frozen. It seemed to SAS that the third team had had little or no 
communication whatsoever with the second, the language team, and 

none with the first, the Government. They felt that they had 

worked for years to no effect at all. The_large experience with 

PILOT Mail had been totally disregarded.| —Jevatuation was STAT 
that the language and the functionality of 2.5 were “abysmal,” 

and CSPO in general has rated the proposed detiverable as 

unacceptable. | 


Those are strong statements, but I agree with them, and I inctude 
a couple of examples to justify them. 


fc) In the-proposed delivery, the user deals with his files only 
through DISPLAY, which js the fundamental operation. This is 
a design decision that has far-reaching 
ramifications -- that the fundamental operations 
functionally deal with a 'screenful’ of information. That 
is, the user's concept of the system should be that he is 
always dealing with a screenful of information. The user can 


delete a file only by inputting ‘delete’ for every record in - 
the file after he has summoned jit to the screen. A file, of 
course, can contain a very large number of records. If a 

user wants to print out a file, he has to go through the 


same exercise with ‘print’: 
eT 


Es A single record at a time Seen a display file may be 
printed. 
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2. Files, parts of records, etc., may not be printed. 
Therefore, no ordering, starting at, etc. 


3.. No COPIES parameter. (i.e., there is no parameter that 
enables a user to print a number of copies.) 


a) Printing of related records which are displayed 
together is possible using the right output forms. 


os Special printer characteristics cannot be specified 
(e.g., font)."* 


fe) The Command Procedures (essentially macros that allow the 
‘user to run sequences of commands with a Single command) 
were to be implemented, but without the features that gave 
them any of the powers needed -- no parameters, for example. 
This is significant because the command procedure would 
allow over-riding of the restrictions discussed in the 
preceding item; a properly written command procedure could 
delete every record in a file with a single simple command. 


Inputting a-command sentence consists of typing a command 
followed by a number of parameters that tell SAFE things about 
how the command is to be executed. The ways that the parameters 
are expressed and ordered or punctuated are called the syntax of 
the language. The syntax makes a large difference in the 
convenience of the Tanguage to the user; compare the following 
three versions of commands that cause a file named BERLIN to be 
printed on a line printer in the usual format named FORML: 


PRINT BERLIN 
PRINT BERLIN, LINEPRINTER, FORMI 


STAT [Internal Memo: "Comments on Customer SUL Letters of 
. Direction," to Distribution, from Hagerbaumer, Breisacher, and 
Almeida, 1/8/82, 82.35656.77-004. 
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PRINT FILE=BERLIN, DEVICE=LINEPRINTER, FORMAT=FORM1 


These command sentences, properly interpreted by a subprogram 

that might be called the Command Interpreter, all mean the same 

thing.# The shortest one, the first, assumes that the user has 
established the lineprinter as the DEFAULT device, and the format 
specification FORM1 as the default format. In fact,[ = | - ~~ STAT 
proposal for the initial SUL syntax seems to look very much like 

the third version, 


Syntax is a complicated subject, but it is obviously crucial to 
the analysts' usage of a system. The third version takes more 
than four times as many keystrokes as the first. The use of 
default values for parameters is a key issue. Sometimes 
parameters have to be entered in a predetermined order, and that 
too can be burdensome to the user analyst. 


There are other restrictions for the user to bear: 


(Gov't question: ) 
"Is the version number incremented when a “file” action 
takes place or when a user calls up an item to be 
edited:(sic) The record is ‘copied’ when routed, will this 
be a new version? Can a user request to see "Tatest" 
versions or must he specifically name the version? When the 
user sees the status of a compose file will he see both item 
name and version number? Does the user have the option of 
replacing a record in lieu of creating a new version? 


STAT ‘[_]response:) 


"Version number is incremented when an item is edited. 
The record is copied when routed. 
Version number is associated with the routed record. 


#0f course, in the proposed deliverable mo such commands would be 
usable at all, since they call for a whale file to be printed in 
a single command. 
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"The user must specifically name the version. 

The user does not see both item name and version number (no 
version mark). The user does not have the option of 
replacing a record in lieu of creating a new version."* 


That means that the user analyst must remember a version number 
and type it in for every file transaction; this would be a 

trivial programming task to get the computer to do. It would to 

my mind be insufferably burdensome to the user. 


The number of restrictions of usage placed on the analyst js far 
too large to be acceptable. Furthermore, of course, they should . 
be checked in the context of reasonable actual usage -- the PMO 
is the ideal place to make sure that the projections are 
reasonable. 


My judgment is that CSPO jis correct in deeming that the extra 

burden and restrictions placed on the user analysts[ - STAT 
proposal are unacceptable in a delivered system. The fact that 

these restrictions derive from a functionality means that they 

are not merely cosmetic, and hence fixable, but instead reflect 

real trouble’ with the design. 


I go on at some length about these topics because they show the 
extent of the communications disconnect between the Government - 
needs and the contractor's interpretation of them. The 2.5 

. release aS proposed at the PDR is not acceptable to the 

STAT Government, [ ipa agreed (as of 2/19/82) to improve its 

' first release. In fact, it is my understanding that] STAT 
recently accepted the entire Government position on the SUL. The 
underlying issues of design in support of it are far less clear, 
however. 


STAT emer ou 24 to Gov't. Comments on Data Base Definition 
Ocument, 1/25/82, at the PDR. 
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2.3 System Design 


One of the most distressing aspects of the current status of SAFE 
is encompassed by the following comments made at the end of the 


PDR: oot . 
a "The system desdas is not com ste “and Srere are sm 
STAT "open areas." Fido, 1/28/82) 
0 "The design must be subject to validations (ibid) 
oO There are real concerns about the "delivery of 2.5 and its — 


viability in the users' hands." (ibid) 


. ; ; STAT 
fe) There are still "substantial system issues ... 


STAT [ por, 1/28/82) 
) The system design is "not yet quite right." (ibid) 


It is now less than 14 months from the scheduled delivery of the 
I0C. The design uncertainties exist in many fundamental functions 
that SAFE must perform, not just in the SUL, discussed above: 


fe) In response to Government comments about procedures for 


Applications Software, ae was? STAT 


‘“We plan to update this level of information as we move into 
detailed design activities. As the detail (sic) design is 
completed, it will be reflected (in the) documentation."* 


STAT 0 | __ the micro sequence array (of MICROSEQ) contains : 
execution.steps and parameters associated with the execution 


STAT [|__| Responses to Gov't comments on Product SHEEP lceetons for 
pplications Software, PDR, 1/26/82, p. B-023. 
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steps. The details of the structure and contents are still 
being defined and should be resolved shortly."# 


STAT 0 5 (ee the units and procedures (in the UIS) are completed 
through detailed design, the detailed work messages and 
subroutine calls will be reflected ..."* 


STAT 0 [ithe detailed design of micro generation is not yet 
. complete ..." (ibid, p. C-022) 


rs) The DeltaData 7260T is still under development; the function 
keys and the documentation are both unciear.* 


TAT 0 | _|emagor SAFE C Design Issues: 


‘ Performance ... 
é User Language 
: Failover(sic)/Recovery 
és Operating Modes 
. External Systems ..."# 
o The only well-established text editor now available in SAFE 


for the proposed delivery is the one provided by the 
DeltaData 7260T terminal; "this is not acceptable as the 


sole text editor for SAFE," [ | CSPO, at the PDR).STAT 


) It is well known in large user-oriented time-sharing systems 
that for most of the time the computer 3s engaged in 
jnterpeting and carrying out commands that essentially have 
to do with organizing and executing editing. As we remarked 
above, there is no editing language well enough specified 


ibid, p. B-031. % 
TAT Responses to Gov't. Comments on UIS Product Specification,STAT 


PDR, 1/26/82, p. C-010 
*Discussions at PDR, 1/28/82. 


ETAT "The SAFE Project," Briefing Yn 
TAT 


1/15/82, chart 25. 
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for inclusion in SAFE. Note that ODP, whose current widely 
used E-language is X-EDIT, has for some time been 
deliberating about the upgrading of that to one more 
sensitive to users' needs and to the increased capabilities 
of modern systems. ODP has now actively embarked on 
developing a moder greestage E-language.* At the PDR, 
there was no evidencecthat Such deliberations were being 
received and considered . 


It should be remarked that these are not the modifications and 
"tuning" that devolve from a system's being put into practical 
operation and then adapted to actual as opposed to projected use. 
No large piece of the SAFE hardware and software has actually. | 
been tested. If there are therefore many fundamentally unfinished 
design issues in SAFE, how realistic is it to-propose an I0C date 
of 3/14/83? I do not think it is realistic at all, and I have not 
found any experienced system analyst who thinks it is, and CSPO 
seems to agree. The point is that SAFE must work reliably -- far 
more reliably than just well enough to correct bugs and to adjust 
some software and rewrite others. 


This skepticism is separate from the problems arising from an 
unacceptablé SUL. I shall return to it later in this section. 


2.4 Failure and Recovery 


Failure (in the[  |documentation jt is inexplicably referred to 
as '‘failover') must be a key concern. Every computer system can 
find new and thrilling ways to fail, but SAFE must protect its 
contents and its users with an unmatched reliability. Since SAFE 
is large and complex, there are many things that can go wrong. 


| __ fitsert has recognized that how to handle failures has still 


*Personal communication, Bruce Johnson, ODP. 
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to be resolved. It is a "Major SAFE C Design Issue."* Failures 
can result from caprices of hardware or software, and since 
nejther has been fully designed, it is not easy to project the 
kinds of failures that will occur. History assures us, however, 
that they will. Not only have methods for handling failures, 
including, of course, recovery from them, not been specified, but 
the design requirements have not even been written except in the 
most general terms. _ a 


2.5 Performance 


The performance of SAFE has been projected with Performance 

Analysis. "Performance Analysis is the primary vehicle ... to. 

verify the software architecture."| «POR, 1/28/82) This STAT 
requires the "appropriate models," and "all tests and benchmarks” 

are performed on the models. The models are built -- indeed they 

have to be built -- using scenarios of projected usage. 


Now that is curious. First, SAS tried unsuccessfully for 18 

STAT: months to persuade| _ fo produce in the form of scenarios their 
notions about what SAFE would actually do for an analyst. Second, 
the scenarios used for the (computer-based) models are not 
current, but are based on data three to four years old. Third, 
the scenarios, and the other assumptions underlying the models, 
take no advantage at all of the experience the Government has had 
with PMO. 


The computer-based projections do not seem unreasonable, given 
the vaguely communicated notions of usage, and supposing that 
there will be no.major changes in architecture. They show that 
the first release will in several ways not satisfy the 
requirements without certain modifications and improvements, 
STAT 


“The SAFE Project, A briefing for] | 
1/15/82, Chart 25. 
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which are now being planned for (of course, they cost extra). In 
the mos favorable projections, with the improvements, the 
margins become smal] and mostly positive. We discuss this point 
further in section 3.4. . 


To a certain extent, that is all to be expected; at the end of 


the PoR,| kai that "the performance problem #s no 
surprise." It must st111 be remembered, however, that the 
projections come from simulations, 


2.6 The Brighter Side 


The CSPO spokesmen profess to be encouraged by what they learned 
at the PDR: ns 


a) ae "The Government does understand the state 
a) @ program ... an ambitious project plan.” 


re) [sd "The level of uncertainty is down," and "we 


need to wrap up the findings, to finalize the design." 


0 [| the senior DIA representative on CSPO, 


. said he was delighted.* 


These optimistic reactions are perhaps de rigeur at the end of a 
POR. Of course, they are understandable after four days of 
grueling reviews and active, sometimes acrimonious, discussions; 
and for many of the Government personnel, I know, the work 
extended far into the night. 


*The PDR was chiefly directed at the C system, for CIA. The D 
system for DIA is different in many ways, but also similar in 
many; the-exact commonalities have not yet been fully determined. 


‘In any case, the D system will be delivered a year or so tater 


than the C. 
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2.7 BUT 


The question to me is whether any optimism is justified at this 
time. If[ _|"management was aimed primarily toward gaining . 
technical control," (p.5 above), what had they been doing for the 
previous 24 months of the contract? Should one believe that they 
could now do better than they had earlier? What had csPO been 
doing for those 24 months? 


I will put it strongly: 


¢) We are less than 14 months from the scheduled delivery. of a 
system that is rated unacceptable. As of 2/19, there is : 
apparently agreement that that delivery will not be made; no 
other delivery date is firm. 


0 The design jis incomplete. 


a) There are many serious jissues to be resolved. 


fo) No substantial piece of programming or coding has been done, 
let along been debugged, tested and used. (Note that this is 


not somebody's fault -- it was specified in the contract.) 


9) The user languages are undefined, so that documentation and 


training are up in the air. 


o 6©F3-r)DnW The known experience with PMO, the only extant system with 
similar requirements and actual performance in the real 
operational environment has been ignored by the contractor, 
or not been made available to him. 


The situation is even worse than that. In some environments, like 
a University or an R&D establishment, users can tolerate 


- inefficiencies and failures -~ some of them might even enjoy the 
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interactions that can help develop a viable and productive 
system. I do not believe that that is true of the CIA or the DIA. 
The DDI analysts (and the other ones too) that I have met are 
jnterested in finding the best ways to help them do their jobs, 
wickedly demanding at best, and they are not at all interested in 
helping a project that for many is alien to their experiences and 
their tastes. That means at least: 


a) SAFE must visibly and immediately work: it must do things 
for analysts that they want done; it must charige how it does 
those things to satisfy different analysts’ different needs, 
even frivolous ones; it must incorporate and exhibit decent 
human factors; it must tolerate human errors gently and 
supportively; it must provide ways to recover from its own 
failures, with no loss (not minimum loss)- of data, and with 

no loss (not minimum loss) of security. 


9) SAFE must work all the time: that means 24 hours a day, 
every day in the year; it must be tolerant and adjusting -- 
for example, if a CPU fails, another one should take its 
place automatically; if a terminal fails, the user has to be 
able to move to another and start over where he left off; if 
a user’cannot remember how to perform a function, it should 
assist him on-line. | 


o To its users SAFE must emanate feelings of adjustability and 
of the possibility of unbounded improvements in response to 
them. If users don't like something, or if they reasonably 
demand another feature, they should feel that SAFE will 
react positively to them; SAFE should also be generating new 
capabilities for testing and applying in the marketplace of 
intelligence analysis and production. 


The 2.5 increment planned for the first delivery is clearly not 


that SAFE, currently negotiated by the Government{ | 
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3.0 OTHER ISSUES 


Nearly two years ago, STAP discussed the SAFE project in memos to : 
the OCI, expressing certain concerns and making certain 


recommendations. Some of those concerns have been alleviated, but 
some are still with us; some of the recommendations have been 
followed, some not. In this section I shall discuss the topics we 
discussed then and their relevance to the current situation in 
SAFE. The key document is "Options for SAFE," STIC 80-002, from | 
STAP. | 


3.1 PILOT SAFE 


STAP made a strong recommendation to upgrade Interim SAFE toward 
a system that would perform as much as possible like what was 
wanted from SAFE.* To a very significant extent that 
recommendation was followed, and the result has been the 
development of the PILOT Mail Operation, PMO. 


PMO has already provided some interesting and relevant lessons. 


According tof head of SAS, whose responsibilities also 


include running PMO, "the usage, number of terms used (in SEARCH 
profiles), the number of operations and commands used by 


‘analysts, are all far higher than we anticipated... and we 
learned some things that we can't do anything about for the early 


SAFE releases."# Here are some other comments: 


"Two months ago we installed the SAFE PMO in (an OSWR Branch 
which) has already obtained significant benefits (including 


*The relevant portions of the STAP Options paper are attached as 


an appendix. 
#Discussion at CSPO, 12/4/81. 
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an) ability to quickly and efficiently create branch filés 
Which are consistent, non-redundant, and accessible to 

a Lisee LHe branch is quickly getting messages on the day they 
were sent rather than the several days (2-3)... via the hard 
copy/registry route"* 


"The A-Specification states that each SAFE mail file will 
receive an average of 17 electrical documents each day. Our 
experience in PMO...suggests that a more reasonable daily 
average of mail received is 105 documents... 


"The A-Specification states that a mail profile will contain 
50 terms on the average, PMO experience suggests ... that the 
average number of terms per profile is 111."# 


"J... imbedded, leading, and trailing "don't cares" are all 
used in the PMO profiles. We see the continued use of the 
"don't care" capability in SAFE at the same high rate as in 
the PMO. 


".,.. 50% of the terms are unique within an 

office. -especially...within the five ... regional offices. 
In soheegiend iy orjented offices, the incidence of unique 
terms is as high as 90%,"* 


Here are some questions# that ought to be answered by Pilot SAFE: 
‘1. What are the usage patterns of naive users? 
2. What are the usage patterns of experienced users? 


3. What needs for modifications of SAFE performance become 


*Memo from Director, NFAC, to Comptroller, CIA, 6/19/81. 
Memo from Chief, SAS, to Director, CSPO, 11/6/81. 

*Memo to Director, CSPO, from Chief, SAS, 1/7/82. 

wFrom STAP Options for SAFE, pp. 6-7. | 


Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2 


2/19/82 OFFICIAL USE ONLY Page 21 
Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2 


manifest from the transition of naive users to experienced 
ones. 


_4. What are the user documentation and training requirements? 
5. What new requirements emerge from the experienced usage? on 


6. How ‘should the services provided by SAFE be modified to take 
advantage of new technology? 


7. What facets of the system can be safely frozen in design 
concept? What parts must be carefully engineered to retain 
flexibility of function and performance? 


It is still true that PMO does not undertake at present to 
explore the other capabilities that SAFE ought to have -- the 
full range of the other three functions, mentioned in the 
footnote on page 3. It must be expected that the projected usage 
of those functions will change with experience too. 


3.2 Human Factors 


STAP recommended a broad human factors study of what it is 
analysts do: 


"Further, ... we reiterate the urgency for a very general human 
factors study of intelligence analysis and the behavior of. 
‘intelligence analysts. "# 


Indirectly, the PMO is providing valuable data itself. But that 
information, of course, valuable as it is, goes but a short way 


towards the goals we were talking of. Our points then are as 


#Options for SAFE, p. 11. Note that the footnote in the quote 
(not cited) itself noted that the point had been made many times 
- pefore. The emphasis was in the original. 
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valid now, I believe. A study initiated two years ago would by 
now be contributing to many of the design decisions being 
currently negotiated by the Government[ | just as the PMSTAT 


_ Study now can. 


‘There are still many human factors issues in SAFE that do not 


reflect the state of the art, and that ought to be examined for 
application in Pilot SAFE. Some of them were mentioned a year and 
half ago in a memo to CSPO from STAP, 8/6/80, Subject: SAFE 
Project User Language Specifications. I intend to review that 
document with the Government people again soon. 


3.3 How Can We be Sure About SAFE? 


The SUL and the functionality of SAFE were the issues on which 

CSPO and the SAS spent disproportionately much time and effort. 

It is clear that those efforts will have been of little avail 
without a lot more work and pressure [Many other parts oF AT 
the SAFE project have in effect been taken on faith. Examples are 


The Data Base Management System (DBMS) 
The InterComputer Communication 
Structured File Processing | 

CPU Capacities 


Note that I am not alleging that there are any deficiencies jn 
the [work on those topics. It is just that the Government 
ought to be in a position to be sure by its own examination of 
the actual machinery and the data on its use. It is possible, 
indeed likely, that the expertise does not exist within CSPO for 
some topics, like DBMS. Given the most crucial nature of the DBMS 
-- it will freeze many future choices and options for a long time 
-- CSPO ought to consider putting together an ad hoc panel of © 


‘experts,* and STAP would be delighted to suggest members. 
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3.4 The Capabilities of SAFE Hardware 


Another worrying aspect of SAFE is the capacity of the CPU 

hardware under reasonable and anticipated operating conditions. | 
Discussions thatTO™~—SSSCSY had with ‘ STAT 
personnel suggested that a single mainframe can handle at most 35° 
users, and 15-20 would be a more reasonable figure for active 

users. The current cspo{ _ |plans call for 100 users per 

mainframe. It should be pointed out that the design is not far 

enough along that that figure of 100 should be regarded as 

frozen; furthermore, a certain amount of work for the user can be 
performed by the DeltaData terminal. As before, this is such a. 

crucial factor that it would be worth an independent check. It 

should also be pointed out that the[ —spardware is tatally STAT 
incompatible with all the other mainframes used by the Agencies.# 

The application of. an independent panel of experts to evaluate 

this problem should be considered. 


3.5 The Intelligence Community and SAFE 


There are some aspects of SAFE-like support that ought to be 
borne in mind, even though they are deliberately not planned now 
to be a part of the SAFE capabilities. Although the present 
concern of Community management must be the short-run decisions 
about the optimal course for SAFE, STAP believes that there are 
some long-run interests that should affect design and utilization 
decisions. . 


SAFE is not now scheduled to be a Community-wide resource, even 
though it will supply services to two Agencies. The D system will 


have ports into COINS# (a very few, just two, I believe); but -- 


*Such ad hoc panels were recommended in the STAP Options paper. 


. #Personal communication, Bruce Johnson, ODP. 


#COINS, Community Online INtelligence System, is a very large 


Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2 


eUELE SU ILEELE, 


2/19/82p proved For Release 2004/07/12  C1A-ROPS4 90933R000500120004-2 


Page 24 


those will be enough to gauge their value. Adding more of 
something that is known to work and work helpfully is always far 
easier than to try to add a completely new feature. STAP believes 
strongly that a SAFE-like capability will form almost the 
cornerstone of this country's intelligence processing in not too 
many decades; and that, furthermore, that capability must enable 
the analysts to take advantage of all the data bases, collection 
techniques, processing powers, and so on, to which the Community 
has access. Actually, CIA has had two COINS terminals in the 
library, I believe, for more than ten years, but users find them 
awkward to use, little training is provided, and there is minimal 
encouragement from management. ) : 


STAP concludes that it is vital to start now to find out how to 
give SAFE these greater capabilities. We suggest again that a 
reasonable way to begin is to provide Pilot SAFE with ports into 
COINS. It will be necessary: 


) To explore and decide on the proper security rules, including 
terminal access, file access, audit trails, and 
communications security. Note that COINS already has its own 
solutions to most of these problems. 


0 To experiment with some form of gateway to control access and 


communications. Note that COINS has had much experience with 
gateways, and for their purposes the wrinkles have been 
jroned out. 


) To begin to study accessing data bases with different access 


languages and protocols. 


network of intelligence and C3I computers with its central node 
and control at NSA. It was started about 15 years ago, and 
originally had only 3 or 4 elements, including the CIA and the 


DIA. The Director has been and : ars carries data cTaT 


at all levels of security. 
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The current plans for SAFE propose that there be no online access 
for SAFE users for data bases outside SAFE, and no access at all 
for outside users into SAFE data bases. We hope that these plans 
will be modified. We suggest, and in fact recommend, that the 
Community commit itself to providing analysts throughout the 
Community with carefully controlled access to all the data bases’ 
jt owns. Obviously some need very circumspect access indeed; but 
we should be learning how to do that right, instead of trying to 
avoid the issue. There have been no studies that suggest that 
that task is somehow inherently impossible. COINS experience 
suggests that it can be done. 


4.0 WHAT HAS AFFECTED SAFE 


The SAFE project is in a bad way. I do not believe that the 
fundamental SAFE capabilities -- represented, say, by Block 1 -- 
can be provided without a very substantial delay beyond the 
current schedule. Even if Block l capabilities are provided, how 
can we be sure that we can keep the other blocks moving ahead, 
and keep on.providing capabilities for the future requirements? 


It would be too simple to think that once the surface 
disturbances have been smoothed over, the SAFE Project can be 
considered rescued and “back on track." The difficulties with the 
language, the SUL, are upsetting enough in themselves, but they 
represent a fundamental failure of the design process, and that 
goes deep indeed. We should not be tempted to believe that once 
that is fixed everything will be ‘all right. 


4.1 The Underlying Causes: History 


; The SAFE project has a long history, and many people have 
contributed to the decisions from which the present has arrived. 
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Some of the decisions seem to have been made almost casually: as 
STAT part of the es a a "Phased Implementation 
Study," to recommend providing e capabilities in steps instead 


of all at once. Therein may be read: . 


"2.1.6 Development Methodology Changes 


Schedule compression resulting from (delays) causes several 
modifications to the development methodology; 


6) To help relieve schedule compression, user language 
prototyping will not be performed."* 


That speaks for itself. Other items slipped in.similar ways. For 
example, the ability for an experienced user to ‘slave' his 
terminal to another's is invaluable in enabling one user to-help 
another with his problems online. Yet an initial requirement for 
such a capability vanished along the way, and was scheduled to be 
jmplemented in segment 6, some unspecified time after the 10C.# 
Similarly, performance tools, simulation and optimization tools 
are delayed indefinitely. | 

In some real sense, we should talk about the causes for this 
situation.only in order to try to correct the inadequacies that 
have led to the present situation. In late 1976, a memo was sent 
to the Chief, Special Projects Staff, ODP, containing comments on 
a paper entitled "SAFE Design Concepts." It stressed the dangers 
of assuming that one could write then the functional requirements 
for a system to be used far in the future; it noted that analysts 
don't know what to ask for if what they are offered is outside 
their experience, and that the system itself, if successful, 
would inherently change patterns of usage. 


: 25X1 
STAT "Phased Implementation Study Final Report,"[ 
25X1 p. 20. 

ee ibid, p. 51, fig. 3-15, under "teleconferencing." 
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The difficulties of SAFE arose through the nature of the needs 
and the processes by which the project was initiated, the 
contractor selected, and the contents of the contract specified. 
A project like SAFE is 1) very large, both in hardware and in 
software, 2) very complex in capabilities, structure and 
function, and 3) interactive with many users. In fact, nearly Cao eee 
large software systems acquired for the Department of Defense = 
suffered from cost and time over-runs, often extreme.* Especially 
because we did not know exactly what the users would be wanting 
to do, it would seem to be advisable not to try to purchase a 
Single system to 'do the job. 


Rather, not too much new should be attempted in each increment: of 
capability or function; and each increment should be evaluated in 
operational conditions and integrated with other components. The 
users -- the analysts -- must be active and contributing 
participants. These éconsiderations should be explored carefully, 
and they suggest a form of acquisition of capabilities that is 
very different from the current SAFE contract. 


4.2 The Underlying Causes: People 


The people who work in the Intelligence Community are by and @ 
large diligent, competent and dedicated. But there ane few among 
them who represent excellefice (but there are some). in the 
Forefront of computer technology; that is exactly one of the 
resources needed in managing and controlling a project like SAFE. 
Indeed, many of us can remember when the Intelligence Community 
in fact supported the state of the art -- Project Lightning, for 
example, moved computer technology ahead five years in one 
stroke, 


Manifestly, the need for real state of the art experts in the 


*The SADWG report, V.E.Jones, ed., ibid. 


Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2 


STAT 


STAT 


STAT 
STAT 


STAT 


STAT 


a:19/82 Approved For Release boob foytte Si -ibes-00933R000500120004-2 


Page 28 


technologies that are so much a part of intelligence these days 
“5s not unique to SAFE3 but there it applies especially, because 
the 

tasks that SAFE is dedicated to are not ‘glamorous’ in the same 
way that space reconnaissance or molecular beams are. One result 
for SAFE is that questions about implementation or design tend to 
be answered by the contractor alone, and then imposed as it were 
on the Intelligence Community. Even if there are experts inhouse, 
the bureaucratic environment | tends to deaden their ambition, 


because apparently so little can be done. 


meee nero PERCE Eh PE 
paneer ae ah nares ee EE ra eae —J 


4.3 The Role of [ 


It is puzzling why| __|seems to have ignored the real needs and 
directions of SAFE. SAFE js not that small a contract, the 
Intelligence Community is not that unimportant, even in the 
context of the military industrial complex has some very 
bright people working for it, some even brilliant. Why were none 


of them assigned to help SAFE? 


STAT 


Even today, other questions remain: if a tough new manager{ | STAT 


is assigned to be Deputy Project Manager for Engineering 
unde why did she accept a PDR date with so many 


undecided items? In fact, why was the proposed user language 

unacceptable? -- both language groups were working for her. If 

the COMPOSE facility is essentially totally undefined as of now, 
had four months to find that out. Of course, 

had thirty. 


The point of these comments is primarily to raise the question of 
the suitability not only of the management structure, but also of 
the particular personnel. If we are going to expect improvements 
from the way they handle SAFE, we must inquire why we 
should expect the people now to do better than they have dane 
already. 
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Competent managers in computer technology have a hard row to hoe. 
They must be aware not only of the people and tools they deal 
with, but also of the bounding technology jtself and its peculiar 
demands and constraints. The managers should have said -- and 
they should be saying now -- that they and their engineers and 
systems analysts need to observe actual intelligence analysts 
working in their tasks, that they need to try operational 
experiments to tune their system components before putting them 
together to form the bigger system. 


These comments stress that jtself must exert more leadership 
in the direction of SAFE than it has been doing. At the PDR I~ 
heard several times in response to questions about SAFE 
capabilities that| as going to meet the requirements. It is 
part off | responsibilities to see that they also provide a 
system that does the job, even if a contract cannot be written to 
say that in contractese. 


4.4 The Role of CSPO 


CSPO also has a tough job. Somehow the technical strengths to 
gauge and evaluate a proposed system have to be produced and 

applied where they will count. As I remarked above, there is 

little reason to be satisfied that the uninspected pieces of 

design for SAFE are in fact acceptable, since so many of the 

people do not seem to understand the operational context that 
SAFE must work in. 


In fact, the contract was initiated with strong restrictions 
jmposed on CSPO's freedom of action. It was determined elsewhere 
that DIA and CIA were both to be customers on the same contract, 
though their needs are very different. Other requirements were 
Jaid on CSPO: contractor communications were to be funneled 
through them, and controlled by them; and the details of the 
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contract were by and large imposed on them with little recourse. 
In retrospect, these restrictions were a significant handicap on 
CSPO's ability to manage and control. 


But CSPO must also be assuming more leadership in where SAFE is 
going as a whole. Leadership is a hard thing to define: it does 
not have to be concentrated in one body, though it often is; it 
does not have to know every detail, though it often knows many; 
ft must have the ability to inspire and excite, to raise levels 
of ambition, to bring large pictures to workers at the level of 
detail. CSPO does not have that kind of leadership. now, and it - 
needs it. — . ; 


5.0 CONCLUSIONS AND RECOMMENDATIONS 


5.1 Discussion 


The problems discussed in this memo are not easy to solve. It 

would be perhaps tempting to regard the money and the effort 

spent so far as an investment that has to be protected with more 
money and more effort. Indeed] sit DIA asserts that STAT 
because things have not been going well we should be prepared to 

do just that: 


"Sayeral major design and technical issues remain... The STAT 
false start and delay ... has (sic) cost at least $5 to $6 
million in effort and a slip of at least 9 months...[ | 
-currently (and, we think, optimistically) projecting an I0C 
for the DIA Block 3 in September 1984... The costs to 
. complete ... are estimated iq project plan to increase STAT 
$12 million ... 
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on behalf of senior management ... does STAT 
now recognize that costs would be increased and the schedule 
extended from the original plan... most new technical 
personnel are just completing the learning curve... 


js working with o modify the operating . STAT 
system software ... both companies also met ... to assess 


the ... yet unannounced nen, spardware eee STAT 


"Conclusions and Recommendations: The difficulties 

encountered ... are not extremely unusual.{ as gone> STAT 
through a difficult learning curve .. but now desperately 

needs continuity... We strongly believe the present project 
management ... should be supported to maintain project 
continuity."* 


The argument can be carried logically to: the worse a contractor 
does, the more he needs extra support, understanding, and 
continuity. The memo emphasizes that the Government does not 
really know where it stands, and that there is real risk for the 
Government whatever it does: 


"Even under this schedule there is considerable concurrent 

software development... it increases the complexity of the 

project and the uncertainty that the whole system will work 
when all software is integrated. The risk here is schedule 

and cost more than performance."# 


But there are obviously limits to the most tolerant contract 
management. Supposing it turns out that most of the elements of 

the system design are badly flawed: are we really prepared to 

keep injecting more and more resources? As we saw above, the 
hardware is not settled, and there are major design issues. How STAT 


*Memo fo 
DIA, 1/12/82, U-O1/RS. 
#ibid, p. 3. 
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can the Government know what the performance can be? 


That is to say, there is some level of assessment of the current 
state of SAFE which would lead to a recommendation of 
cancellation of the contract. The consequences would be deeply ; 
disturbing and traumatic to the Intelligence Community. The CIA 
has been counting on SAFE strongly enough, but there are other 
viable efforts that can provide some avenues for support for the 
analysts. But DIA lacks those other avenues almost entirely, and 
interruption in progress toward SAFE-like capabilities would be 
catastrophic. SAFE has been ten years in its planning and 
organizing; one would not like to have to wait another ten. 


Since a profound evaluation of the actual current state of SAFE 
seems to be essential to rational decisions, we recommend that it 
be done. It would seem appropriate to have that evaluation done 
by experts who have not participated jin the operation or 
management of the project so far, even if merely to add weight to 
their credibility. 


We can see a range of possibilities: 


1. ° The evaluations are strongly negative; in spite of the 

- negative consequences, the contract should be cancelled, the 
pieces picked up as well as can be done, and the efforts 
restarted. It must be remembered that the Community's needs 
are still there. It is possible that commitments have been 
made that require salvage of the work contracted for; the 
labor needed for that might be large, but it should not be 
allowed to impede the recommended enhancements of the 
ongoing PMO and other expressions of the drive toward the 
‘Capabilities that the analysts need. 


2. ° The evaluations are strongly positive; once the SUL problems 
have been solved, there is every reason to expect that the 
design will lead to an excellent and satisfactory system, 
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after the Blocks are completed. The extra delays and costs 
Will still have to be allowed for, of course. 


There are enough faults and uncertainties uncovered in the 


- evaluations that the Government cannot justify paying for 


yet another learning curve, and cannot trust the current 
procedures to produce what is needed. What we have to do 
then is more in the nature of getting what we can in the 
nature of a system good enough to experiment with, to plan 
with, and to use as a sort of laboratory facility. In that 


event, the initiation of parallel efforts, like a Pilot SAFE 


built around PMO, could be vastly aided by the experimen- 
tation that would be made possible. 


It seems to me much the most likely that the third. possibility 
Will obtain. We might well hope for something better, but the 


Government ought not to count on it. The conclusions and~ 


recommendations below are based on the supposition that the third 


one 3s indeed valid. 


5.2 


a 


Conclusions 


The SAFE project is in very deep trouble indeed. The DCI 
needs to know what the status of SAFE is now, and what can 
reasonably be expected of it given any of the possible 
decisions that may be made, 


The big obvious areas of concern,| are: 


The SAFE User Language 
Many Aspects of System Design 
Simulation, Benchmarking, Performance Analysis. 


It is important that those areas be checked by impartial ~ 


experts. For example, the failure of SUL represents 
fundamental problems in the underlying system design. 
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Other areas of concern are: 
Data Base Management System 
InterComputer Communication 
Structured File Processing 
CPU Capacities. 


There is no reason to trust the| _ |proposed design as being 
suitable for the applications. These areas especially need 
inspection and evaluation by experts who work for neither 
CSPO 


The contractor has never really understood the problems of 
jntelligence processing and analysis, as represented by the 
proposed solutions. to the implementation of SAFE. It is 
extremely important that the contractor have that kind of 
understanding, and that the work reflect it. 


Increment 2.5 is not acceptable as a deliverable. The least 
that can be accepted is the full Block 1. 


The earliest reasonable date for the delivery of Block 1 is 
late in calendar 84; that presumes that much of the current 
underlying design turns out to be acceptable and sound. {t 


- should be expected that the extra time needed for the proper 


delivery of Block 1 will need proportionate funding. 


If the current underlying design for SAFE needs drastic 
revision, then it must be accepted that much of the 
investment thus far in SAFE has been lost. 


The design of SAFE must be undertaken with continuing 


contact between the engineers and the analysts, so that 
there is'a real appreciation of the operational contexts in 


which SAFE must work. 
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Recommendations 


STAP recommends that if the SAFE contract is to be continued, 
then: 


ie 


The operation of Pilot Mail be continued and extended; the 
Inclusion of some of the other functions of SAFE should be. 
considered very strongly. This operation should be regarded 
as an integral part of the SAFE development process. Systems 
analysts and engineers responsible for design decisions 
should be responsible for being aware of experience with 
pilot SAFE. 


An ad hoc panel be set up to evaluate the underlying design 
of SAFE as it currently is. It is jmportant that this be 
done quickly, and that the panel provide an assessment by 
this spring.. The panel should be composed of outside 
experts. STAP will be happy to lend assistance. 


If the panel's assessment is favorable enough, the contractor 
be tasked to provide an integrated design for Block 1; this 
design should allow for further blocks and features perhaps 


even beyond. those specified in the original requirements. 


If the panet's assessment is favorable enough, a number of 


Government/contractor teams be established to review the 
existing set of user requirements to ensure that the 


contractor does indeed have a thorough understanding of each 


requirement--what it isl fis meeting it, how the 
intelligence analysts would use it, and so on. ‘Establishing 
these teams should be spelled out in the contract. 


Both the contractor and CSPO be made responsible for 
providing a greater degree of technical expertise and 


‘leadership in the development of SAFE. 
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Alternatively, the SAFE contract should be terminated. 
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GLOSSARY OF ACRONYMS 


COINS 
“COTR 
“CPU 

CPO 

DBMS 

DDI 


I¢ 
Ioc 


Community Online INtelligence System 


Contract Office's Technical Representative é 


Central Processing Unit (of a computer) 


Consolidated SAFE pRotedt Office 


Data Base Management System 
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Deputy Director for Intelligence -~ used as shorthand 


For Intelligence Directorate; used to be NFAC, 
Foreign Assessment Center 


Intelligence Community, as in IC Staff 
Initial Operating Capability 

Office of Central Reference 

Office of Data Processing 

Preliminary Design Review 

Pilot Mail Operation 

Support for the Analysts' File Environment 
Systems Analysis Staff 

Science & Technology Advisory Panel 


SAFE User Language 


eee 


User Interface Working Group (of CSPO) 


User Language Specification 
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STAP OPTIONS FOR SAFE 


1. Introduction 


On 18 March 1980 we forwarded to you eight questions 
regarding the future evolution of the SAFE system and the 
relation of CIA SAFE to other community systems (See. 
Appendix I). In this report we propose various actions 
which, if implemented, could yield a more valuable community 
asset in the long run. Because of the size of the system and 
its complexity, a delay (6 months to two years) in IOC can 
be anticipated. Productive use of this delay time can be 
made, as we discuss in subsequent sections. 


In our examination of SAFE, we were impressed with the. 
need. for a community manager of the ADP-communication 
systems. None now exists and SAFE is not being integrated 
into an overall community architecture. As a result the 
incremental value of SAFE will be less than it could be. 
Even without a community manager, the future capabilities of 
SAFE could be strengthened and possible steps in that 
Girection are described in Section 2. The longer term 
questions of technical direction of the overall Intelligence 
Community ADP-communication systems will be the subject of 
additional STAP analysis and should be considered a separate 
issue from that of SAFE. ; 


The evolutionary development of SAFE will require 
analysis of how the community uses SAFE. Sections 3 and 4 
describe means by which such analysis could proceed. As SAFE 
comes on line, ,it will be essential for future planning to 
evaluate its usefulness. A possible means for evaluation is 
described in Section 5. : 


Finally, we believe. that a rich body of experience in 
systems similar to SAFE exists and could be beneficially 
applied to enhancing SAFE's capabilities. To this end, we 


OFFICIAL USE ONLY 


Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2 


% a . OFFICIAL USE ONLY 


Approved For Release 2005/07/12 : CIA-RDP84-00933R000500120004-2 


‘propose in Section 6 establishment of an Advisory Council on 
Technology for SAFE. “ 


This report primarily examines CIA SAFE and its use by 
the Intelligence Community. Our emphasis on CIA SAFE derives 
from the fact:that NFAC stands to. benefit greatly froma 
truly operational SAFE. DIA needs center on the accuracy, 
maintenance ‘capability and general utility of their large 
encyclopedic files. These files require restructuring and 
improved maintenance capability as well as a high level of 
concurrency in use. Our analysis focuses on the analyst 
Support function of SAFE; this function is of secondary 
importance to DIA. DIA's requirement can more easily be met 
than CIA's. Thus while wé propose a “true pilot SAFE" for 
CIA, such an activity should not impede the development of 
the DIA SAFE. Indeed, the lessons learned in the "true pilot 
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SAFE" will be of use in the evolutionary development.of the _ 


DIA system. 


In outlining the options for the future, we are fully 
aware of and appreciate the ‘concerns of the SAFE management 
office and the Office of Central Reference. -The steps 
outlined below will delay the scheduled delivery of an 
operational system, but we believe that the present schedule 
cannot be met since such critical items as command language 
and central hardware have not yet been decided upon. The 
delay we anticipate can be put to use to obtain operational 
experience on a "true pilot SAFE." The delays whether 
anticipated or not will cause problems with OMB and Congress 
and these should be recognized now. 


2% pienengepenaws the SAFE Management 


Strengthening of the community management of SAFE is 
essential if it is to become effective in satisfying its 
prescribed functions, and be capable of expanding flexibly 
and responsibly to aid the entire community. 
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provide full-time support with undivided loyalties. They 
must, of course, have the rank and authority to deal with 
their contractor counterparts. 


It is also likely that some outside expert advice would 
be helpful; and for 3) above, a standing Advisory Council on. 
Technology (ACT) would seem most fitting. It is not clear 
whether the ‘council should restrict its considerations to 
SAFE, or should in fact ultimately deal with a broader range 
of technological problems. These questions are amplified 
below in Section 6. 


3. A True Pilot SAFE 


Interim SAFE was initiated in 1974 as a-set of. 
capabilities on a 370/158 run by ODP. Four main capabilities 
were sought for the original SAFE project, and they © are 
still valid today: 

1) A mail/message/distribution system ~ 
2) Private files available on-line for analysts 


3) Public files available on-line for analysts 


4) On-line facilities for read, edit, write, and 
document production 


These four have to be somewhat extended and modified in 
detail to match either today's purported goals or the real 
needs that underlie the requirements for the system. / 


The chief uses made of interim SAFE were: 


1) To provide limited experience for certain 
analysts in order to survey their expressed needs. 
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2) To demonstrate the capabilities to management 


echelons, in order to help with the budget and funding 
processes, 


3) ‘To derive certain specifications that might 
Porte as guides for the actual’ SAFE. system 
specifications. 7 oa ; 


4) °- To illustrate+--the --capabilities in’ .the 
intelligence environment to possible proposed - SAFE 
contractors. . 


It is important to observe that Interim SAFE was never 
used as a__pilot system as that term is used in 
engineering--that is, to provide experience with a small 
system whose performance is operationally projected- to be 
what the final system ought to be. In practice, of course, 
the . pilot system serves in engineering to modify. 
requirements and specifications in both usage and 
engineering. 


In Interim SAFE, we were informed that in general 
Statistics about usage were not gathered because they would 
be "not representative." 


The questions that ought to be answered by a true pilot 
SAFE include: 


1) What are the usage patterns of naive users? 


2) What are the usage patterns of experienced 
users? 


. 


3) What needs’ for modifications of SAFE 
periormance become manifest from the transition of 
naive users to experienced ones? 


4) What are the user documentation and training 
requirements? 


=a} What new requirements emerge from the 
experienced usage? 3 
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6) How should the services provided by SAFE be 
modified to take advantage of new technology?* 


7) What facets of the system can be safely frozen 
in design concept? What parts must be carefully 
engineered to retain flexibility of function and 
performance? eo! 


Only a few of these questions can be dealt with through 


the current Interim SAFE. But there has been a great deal of 


experience elsewhere in ‘systems similar in nature and size - 
to SAFE; we note the unique aspects of much of the 

intelligence environment, which is why a true pilot SAFE is 
needed. But if . advantage is taken of the continuing 
experience of these other systems, it is likely that the 
process of initiating a pilot SAFE and interacting with it 
to guide final SAFE development can be speeded up. 


The questions above that seem at first glance to be 
uniquely answerable by pilot SAFE are 1, 2, 3, and 5. These 
have to do with analyst usage of the tools and capabilities 
provided to them. We are therefore suggesting that a first 
step is to. start collecting systematic longitudinal data on 
analyst usage of the current Interim SAFE, continuing during 
a conversion to a true pilot SAFE. 


There is little doubt that the general capabilities 
sought can be provided on a small scale by any of a large 
number of current installations outside the community, as 
well as inside. At least four members of STAP have had wide 
experience with such systems. Such systems also provide some 
of the extra capabilities that are not now planned as part 
of SAFE, 4 but that are considered highly desirable, 
including:! os 


1) collaborative capabilities, whereby several 


analysts can simultaneously and remotely collaborate on 
the same task. 


re ee eee ee ee eee ee ee Se Cee tee me OS Oe moe 


*Por example, good split-screen graphics terminals can 
take advantage of more powerful editing capabilities than 
previous terminals. 
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2) general communications access to remote files 
over community or common carrier. lines;. this is 
relevant to making SAFE truly a community-wide 


resource, as is being strongly urged by certain members 
of the Ic staff. . 


3) new. evolutionary user languages, including the 
capability for user on-line control of multiple jobs 
simultaneously. T oee 4 oo 


The actual. implementation of pilot SAFE can therefore 


be handled in several ways: © 


1) Use the interim SAFE now running in ODP. 
Upgrade its capabilities, installing - the above desired 
new capabilities, aiming at an integrated single system 
for continuing development. oe 


2) Use.an existing system from outside, but 
installed at the CIA, using outside contractor support 
as well as in house personnel. a 


3) Use an existing system at an outside 
contractor's installation, sending analysts on TDY to 
provide the usage, etc. The difficulties involved in 
this option are obvious, and it is included only for 
the sake of completeness. 


The advantages of the first option are that interim 
SAFE is now running here, that both users and system 
personnel are familiar with it, and that it performs some of 
the needed capabilities already in the desired intelligence 
environment. The disadvantages are that some , system 
reprogramming will. be necessary to bring it to state of the 
art, and that this will likely need system architecture and 
programming resources beyond what is available here. 


The advantage of the second option is that such a pilot 
system is nearly an off-the-shelf item, and could be running 
in the CIA fairly quickly. On the other hand, the special 
requirements of the community environment would enormously 
delay the user population as well as the systems and 
programming personnel at the CIA. 
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In summary, we strongly suggest option 1, i.e., use the 
current interim SAFE. A conscientious effort toward the kin 
of pilot SAFE being discussed here entails:. 


1) Increased technological resources available in 
house. This should include some contractor 
STAT personnel, plus some contractor personnel, to 


rage 49 


keep the former honest, as it were. A total of six ~ 
people for the first eight months would be needed, and | 


then perhaps dropping to four on a continuing basis. 


: 2) Enlarging the user population and the user 
studies. This is so important, and has so many 
ramifications, that is the subject of a separate 
Section 4. . 


3) Establishing a direct COINS link that. will 


enable study and experimentation by analysts in 


community-wide access and retrieval. 


4) Initiating data gathering and analysis of usage 
pattern and changes in usage patterns by analyst users 
(see also Section 5) es 


5) Testing prepared user languages and other tools 
as soon as possible. 


4. The Users of SAFE 


There are two main reasons why the population of users 
of Pilot SAFE must be enlarged and modified from that of 
users of Intermim SAFE: 


1) We need to find out answers to what users do, 
how they do it, and how they change. 


2) We need users to find. out what SAFE can do 
for them, and how it can be responsive to 
their developing needs. 

The first dictates the initiation of the continuing study of 
user patterns of usage that we have already mentioned. An 


IBM study of a somewhat different user community illustrates 
the approaches and the attacks that might be considered; it 
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